Evolutionary Algorithms for Reinforcement Learning

There are two distinct approaches to solving reinforcement learning problems, namely, searching in value function space and searching in policy space. Temporal difference methods and evolutionary algorithms are well-known examples of these approaches. Kaelbling, Littman and Moore recently provided an informative survey of temporal difference methods. This article focuses on the application of evolutionary algorithms to the reinforcement learning problem, emphasizing alternative policy representations, credit assignment methods, and problem-specific genetic operators. Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications

The conformal mapping of the interior of the unit circle onto the interior of a class of smooth curves

Outer and inner curved plates and shell shapes optimized by conformal mappin

A highly optimized vectorized code for Monte Carlo simulations of SU(3) lattice gauge theories

New methods are introduced for improving the performance of the vectorized Monte Carlo SU(3) lattice gauge theory algorithm using the CDC CYBER 205. Structure, algorithm and programming considerations are discussed. The performance achieved for a 16(4) lattice on a 2-pipe system may be phrased in terms of the link update time or overall MFLOPS rates. For 32-bit arithmetic, it is 36.3 microsecond/link for 8 hits per iteration (40.9 microsecond for 10 hits) or 101.5 MFLOPS

Margins within margins?: voices Speaking through a study of the provision of an educational program for the children of one Australian show circuit

This paper examines the tactics used by the Showmen's Guild of Australasia in successfully lobbying for the development of a distance education program for their children. The Guild is considered to be a 'marginalized' group, meaning members have less access to wealth, power, and status. Since 1930, members of the Showmen's Guild and their families have traveled from town to town providing agricultural and equestrian shows. Despite the diversity of backgrounds and experiences among people connected with the show circuit, the Guild is highly organized and has been politically active. Informal sanctions have been effective in enforcing group discipline and in presenting the image of a single body of opinion. In addition, investment in sophisticated machinery and technology has resulted in show people having the financial resources to buy homes and have a political voice via more 'normalized channels'. Although members learn early that they are a marginalized group and are perceived as different from the mainstream, the group maintains close ties and often celebrates its difference. Implications for educational program development center on the goals of educational programs designed for disadvantaged groups, and the status of other marginalized groups and their efforts to contest their marginalized status. (LP

Two-electron bond-orbital model, 2

The two-electron bond-orbital model of tetrahedrally-coordinated solids is generalized and its application extended. All intrabond matrix elements entering the formalism are explicitly retained, including the direct overlap S between the anion and cation sp3 hybrid wavefunctions. Complete analytic results are obtained for the six two-electron eigenvalues and eigenstates of the anion-cation bond in terms of S, one-electron parameters V2 and V3, and two-electron correlation parameters V4, V5 and V6. Refined formulas for the dielectric constant and the nuclear exchange and pseudodipolar coefficients, as well as new expressions for the valence electron density, polarity of the bond and the cohesive energy, are then derived. The theory gives a good account of the experimentally observed trends in all properties considered and approximate quantitative agreement is achieved for the pseudodipolar coefficient

Low-profile and wearable energy harvester based on plucked piezoelectric cantilevers

The Pizzicato Energy Harvester (EH) introduced the technique of frequency up-conversion to piezoelectric EHs wearable on the lateral side of the knee-joint. The operation principle is to pluck the piezoelectric bimorphs with plectra so that they produce electrical energy during the ensuing mechanical vibrations. The device presented in this work is, in some ways, an evolution of the earlier Pizzicato: it is a significantly more compact and lighter device; the central hub holds 16 piezoelectric bimorphs shaped as trapezoids, which permits a sleek design and potentially increased energy output for the same bimorph area. Plectra were formed by Photochemical Machining of a 100-μm-thick steel sheet. To avoid the risk of short-circuiting, the plectra were electrically passivated by sputtering a 100 nm layer of ZrO2. Bench tests with the steel plectra showed a very large energy generation. Polyimide plectra were also manufactured with a cutting plotter from a 125μm-thick film. Besides bench tests, a volunteer wore the device while walking on flat ground or climbing stairs, with a measured energy output of approximately 0.8 mJ per step. Whereas most of the tests were performed by the traditional method of discharging the rectified output from the EH onto a resistive load, tests were performed also with a circuit offering a stabilised 3.3 V supply. The circuit produced a stable 0.1 mA supply during running gait with kapton plectra

Transition-metal interactions in aluminum-rich intermetallics

The extension of the first-principles generalized pseudopotential theory (GPT) to transition-metal (TM) aluminides produces pair and many-body interactions that allow efficient calculations of total energies. In aluminum-rich systems treated at the pair-potential level, one practical limitation is a transition-metal over-binding that creates an unrealistic TM-TM attraction at short separations in the absence of balancing many-body contributions. Even with this limitation, the GPT pair potentials have been used effectively in total-energy calculations for Al-TM systems with TM atoms at separations greater than 4 AA. An additional potential term may be added for systems with shorter TM atom separations, formally folding repulsive contributions of the three- and higher-body interactions into the pair potentials, resulting in structure-dependent TM-TM potentials. Towards this end, we have performed numerical ab-initio total-energy calculations using VASP (Vienna Ab Initio Simulation Package) for an Al-Co-Ni compound in a particular quasicrystalline approximant structure. The results allow us to fit a short-ranged, many-body correction of the form a(r_0/r)^{b} to the GPT pair potentials for Co-Co, Co-Ni, and Ni-Ni interactions.Comment: 18 pages, 5 figures, submitted to PR

Inactivation by UV-Radiation and Internalization of Coliforms and Escherichia coli in Aquaponically and Hydroponically Grown Lettuce

Recent studies have shown that coliform bacteria, specifically Escherichia coli O157:H7, can internalize in leafy green produce. Therefore, a risk exists for lettuce being grown in systems using mixed water sources. Additionally, Ultra Violet light radiation has been shown to inactivate coliforms in water. For this reason, an experiment was conducted at West Virginia University\u27s aquaculture facility to determine the effect of UV-radiation in water supplying an aquaponic channel with nutrients, and to assess the risk of coliform and E. coli internalization. In this experiment, the spring fed, flow-through aquaponic system was receiving runoff from cattle pastures as a source of coliform bacteria. An inline UV-light system was randomized to treat water going to half of the hydroponic channels growing lettuce downstream from the raceways containing fish. Water going into and leaving the eight channels was sampled for 6 weeks in addition to the upstream water (before and after the fish raceways). Additionally, lettuce samples were aseptically collected during the time of harvest. Water samples were plated directly on 3-M Petrifilm EC (E. coli/ coliform) as well as on M-endo agar using a membrane filter. Incubation was done at 35°C for 48h for both media. Lettuce samples were surface sanitized, homogenized, and directly plated on M-endo agar. The mean coliform count of all untreated water was 1.93 x 101 CFU/ml. A 3-4 Log reduction in total coliforms for treated water entering the channels was observed. There were no detectable levels of E. coli in the treated water entering the channels; the mean concentration of untreated water was 0.69 CFU/ml. Additionally, internalized coliforms were not detected in the lettuce samples. Although, black colonies did grow on the media, they tested negative in a confirmation test; discoloration was most likely due to the decomposition of the lettuce particles within the samples. This study demonstrates the effectiveness of coliform inactivation by UV-radiation in water and suggests that coliform/E. coli internalization in lettuce may be a conditional phenomenon.;In efforts to verify the possibility of internalization in lettuce grown in water, as well as its dependence on conditional factors, a follow-up study was conducted at West Virginia University. The objective of this study was to determine the level in which internalization of E. coli occurs in lettuce as well as the effect plant root injury may have on it within a hydroponic system. A hydroponic system was set up to grow lettuce in water inoculated with Escherichia coli O157:H7. The inoculated water was kept at a concentration around 105 CFU/ml of E. coli O157:H7 for the duration of this study. Half of the plants were randomized to receive 3 consecutive treatments of damage to the root tips. Edible portions of the lettuce were then harvested to be analyzed. The lettuce was surface-sanitized, homogenized, and filtered using a bag mixer. Sample dilutions were prepared and inoculated on 3-M Petrifilm EC. Internalization was observed at a mean of 1.66 x 101 CFU/g in both treated and untreated plants and as high as 4.90 x 104 CFU/g in one treated sample. Although there was no significant difference between treatment and the control, it should be stated a difference was observed. Taking into account the sample size used, significance may be seen if this were repeated on a larger scale. The variation within the samples may also suggest other contingent conditions may be a factor

Exploring the relationship between a fluid container\u27s geometry and when it will balance on edge

At some point while consuming a beverage, many people will idly try to balance its container on edge. The act itself is physically straightforward, merely involving the system\u27s center of mass and achieving a static equilibrium between the opposing torques caused by gravity and the normal force between the container and the surface on which it balances. Further analysis of the act, however, illuminates the richness of the exercise. These nuances are apparent even in simplified two-dimensional models because of the depth of the relationship between a container\u27s geometry and achieving balance. The purpose of such analysis is threefold: first, when considering a rectangular container, to determine the relationship between the angle at which it balances and the amount of fluid in the container; second, to consider a massless analogue to a standard twelve-ounce aluminum can which balances at a fixed angle and observe the interplay between the various parameters of that container\u27s geometry and balance; and finally, to revisit the aluminum can model, this time considering its mass relative to the fluid\u27s, and recover the familiar behavior observed when balancing real-world beverages in aluminum cans